WEST Refine Search 



Page 1 of 1 



Refine Search 



Search Results - 



Terms 


Documents 


L3 and sort$ and intervals and rankS and scor$ 


5 



US Pre-Grant Publication Full-Text Database 

US Patents Full-Text Database 

US OCR Full-Text Database 

EPO Abstracts Database 

JPO Abstracts Database 

Derwent World Patents Index 

IBM Technical Disclosure Bulletins 



Database: 



Search: 



L4 



mm 




Search History 



DATE: Wednesday, March 29, 2006 Printable Copy Create Case 

Set Name Query Hit Count Set Name 

side by side result set 

DB=PGPB. USPT, USOC.EPAB.JPAB,DWPI.TDBD; PLUR=NO; OP=OR 

L4 L3 and sort$ and intervals and rank$ and scor$ 5 L4 

LI L2 and (evaluatS or estimatS or determinS) 155 L3 

12 LI and decision same making • 157 L2 

LI 706/46.ccls. 521 U 

END OF SEARCH HISTORY 



http://westbrs:9000/bin/cgi-bin/PreSearch.pl 
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Search Results - 



Terms 


Documents 


L5 and decision same making and (evaluat$ or estimat$ or determin$) and sort$ and 
intervals and rank$ and scorS 


11 



US Pre-Grant Publication Full-Text Database 

US Patents Full-Text Database 

US OCR Full-Text Database 

EPO Abstracts Database 

JPO Abstracts Database 

Derwent World Patents Index 

IBM Technical Disclosure Bulletins 



Database: 



Search: 



L6 



. Jntejjupt-^t 



Search History 



DATE: Wednesday, March 29, 2006 Printable Copy Create Case 



Set 

Name Query 
side by 
side 

DB=PGPB,USPZUSOC,EPAB,JPAB,DWPlTDBD; PLUR=NO; OP^OR 



Hit Set 
Count Name 
result set 



L6 


L5 and decision same making and (evaluatS or estimat$ or determinS) and 


11 


L6 


sort$ and intervals and rank$ and scor$ 


L5 


706/45-47.ccls. 


1515 


L5 


L4 


L3 and sort$ and intervals and rank$ and scorS 


5 


L4 


L3 


L2 and (evaluatS or estimatS or determin$) 


155 


L3 


L2 


LI and decision same making 


157 


L2 


Li 


706/46.ccls. 


521 


LI 



END OF SEARCH fflSTORY 



http://westbrs:9000/bin/cgi-bin/PreSearch.pl 
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Refine Search 



Yovir wildcard search against 10000 terms has yielded the results below. 
Your result set for the last L# is incomplete. 
The probable cause is use of unlimited truncation. Revise your search strategy to use limited truncation. 

Search Results - 



Terms 


Documents 


L4 and selectS 


5 



US Pre-Grant Publication Full-Text Database 

US Patents Full-Text Database 

US OCR Full-Text Database 

EPO Abstracts Database 

JPO Abstracts Database 

Derwent World Patents Index 

IBM Technical Disclosure Bulletins 



Database: 



Search: 




Search History 



DATE: Wednesday, March 29, 2006 Printable Copy Create Case 
Set 

Name Query 
side by 
side 

DB=PGPB, USPZ USOQEPAB,JPAB,DWPlTDBD; PLUR^NO; OP=OR 



Hit Set 
Count Name 
result set 



LZ 


L4 and select$ 


5 


L7 


L6 


L5 and decision same making and (evaluatS or estimatS or determin$) and 
sort$ and intervals and rank$ and scor$ 


11 


L6 


L5 


706/45-47.ccls. 


1515 


L5 




L3 and sort$ and intervals and rank$ and scor$ 


5 


L4 


L3 


L2 and (evaluat$ or estimatS or determinS) 


155 


L3 


L2 


LI and decision same making 


157 


L2 


LI 


706/46.ccls. 


521 


Li 



END OF SEARCH HISTORY 



http://westbrs:9000/bin/cgi-bin/PreSearch.pl 
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Search Results - 



Terms 


Documents 


L14 and @pd<"2003 1029" 


• 417 



US Pre-Grant Publication Full-Text Database 
US Patents Full-Text Database 
US OCR Full-Text Database 
EPO Abstracts Database 
JPO Abstracts Database 
Derwent World Patents Index 
IBM Technical Disclosure Bulletins 




Search History 



DATE: Wednesday, March 29, 2006 Printable Copy Create Case 



Set 

Name Query 

side by 
side 


Hit 
Count 


Set 
Name 
result 

set 


DB=PGPB,USPT,USOC,EPAB.JPAB,DWPI,TDBD; PLUR=NO; OP=OR 






L16 


L14and@pd<"20031029" 


417 


L16 


L15 


L14@pd<"20031029" 


32788637 


L15 


L14 


LI 3 and decision and preferences 


706 


L14 


L13 


questions and answerS and choiceS and evaluatS and scor$ and (sort$ or 

classS) 


2533 


L13 


L12 


Lll and sorting near preferences 


2 


L12 


Lll 


decision same making and choices 


4441 


Lll 


Lie 


L9 and @pd<"2000 1231" 


16 


Lie 


L9 


L8 and processor and memory and computer 


190 


L9 


L8 


decision same making and (evaluatS or estimatS or determinS) and sort$ 
and intervals and rankS and scor$ and (preferencS or choice or choos$) 


311 


LS 


L2 


L4 and selects 

L5 and decision same making and.(evaluatS or estimatS or determinS) and 


5 


H 



http://westbrs:9000^in/cgi-bin/PreSea^ch.pl 
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L6 sorts and intervals and rank$ and scor$ 11 

Li 706/45-47.ccIs. 1515 L5 

L4 L3 and sort$ and intervals and rankS and scorS 5 L4 

L3 L2 and (evaluatS or estimatS or determinS) 155 L3 

L2 LI and decision same making 157 L2 

Li 706/46.ccls. 521 U 

END OF SEARCH HISTORY 



http://westbrs:9000/bin/cgi-bin/PreSearch.pl 
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Search Results - 



Terms 


Documents 


LI 7 and choic$ and scor$ 


19 



US Pre-Grant Publication Full-Text Database 
US Patents Full-Text Database 
US OCR Full-Text Database 
EPO Abstracts Database 
JPO Abstracts Database 
DenA/ent World Patents Index 
IBM Technical Disclosure Bulletins 




Search History 



DATE: Wednesday, March 29, 2006 Printable Copy Create Case 



Set 

Name Query 

side by 
side 

DB=PGPB,USPT,USOC,EPAB,JPAB.DWPI,TDBD; PLUR=NO; OP=OR 
L18 L17 andchoic$ and scor$ 

therapeutic near treatment near regimens 
L14 and @pd<"2003 1029" 
L14@pd<"20031029" 
LI 3 and decision and preferences 

questions and answer$ and choice$ and evaluat$ and scor$ and (sort$ or 
class$) 

LI 1 and sorting near preferences 
decision same making and choices 
L9and@pd<"20001231" 
L8 and processor and memory and computer 

decision same making and (evaluat$ or estimatS or determinS) and sort$ 
and intervals and rank$ and scor$ and (preferencS or choice or choosS) 



L17 
L16 
LIS 
L14 

L13 

L12 
Lll 
LIO 
L9 

L8 



Hit 

Count Kame 
^^^"^ result 
set 



19 L18 

103 L17 

417 L16 

32788637 US 

706 L14 

2533 L13 

, 2 L12 

4441 LH 

16 UO 

190 L9 

311 L8 



http://westbrs:9000/bin/cgi-bin/PreSearch.pl 
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L7 


L4 and selects 


5 


LI 


L6 


L5 and decision same making and (evaluatS or estimatS or detenmn$) and 


11 


L6 


siort$ and intervaHl and rankt and Qrnrt 








T r 


T /I 


L3 and sort$ and intervals and rank$ and scor$ 


5 


T A 

L4 


L3 


L2 and (evaluatS or estimatS or determin$) 


155 


L3 


L2 


LI and decision same making 


157 


L2 




706/46.ccls. 


521 


LI 



END OF SEARCH HISTORY 



http://westbrs:9000/bin/cgi-bin/PreSearch.pl 
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Search Results 



Terms 


Documents 


therapeutic near system and choic$ and scor$ and rank$ 


35 



Database: 



US Pre-Grant Publication Full-Text Database 

US Patents Full-Text Database 

US OCR Full-Text Database 

EPO Abstracts Database 

JPO Abstracts Database 

Derwent World Patents Index 

IBM Technical Disclosure Bulletins 



Search: 



L19 




Jntefruptil 



Search History 



DATE: Wednesday, March 29, 2006 Printable Copy Create Case 



Query 



Set 
Name 
side by 
side 

DB=PGPB,USPT,USOC.EPAB.JPAB,DWPI,TDBD; PLUR=NO; OP=OR 



Hit 
Count 



result 
set 



L19 


therapeutic near system and choic$ and scor$ and rank$ 


35 


L19 


L18 


L17 and choic$ and scor$ 


19 


L18 


L17 


therapeutic near treatment near regimens 


103 


L17 


L16 


L14 and @pd<"2003 1029" 


417 


L16 


L15 


L14@pd<"20031029" 


32788637 


L15 


L14 


LI 3 and decision and preferences 


706 


L14 


L13 


questions and answerS and choiceS and evaluatS and scor$ and (sort$ or 

classS) 


2533 


L13 


L12 


Lll and sorting near preferences 


2 


L12 


Lll 


decision same making and choices 


4441 


Lll 


LIQ 


L9and@pd<"20001231" 


16 


LIO 


L£ 


L8 and processor and memoiy and computer 

decision same making and (evaluat$ or estimatS or determinS) and sort$ 


190 


L9 



http://westbrs:9000/bin/cgi-bin/PreSearch.pl 
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T Q 


and intervals and rankS and scorS and (preierencS or choice or choosS) 


311 


L8 


LI 


L4 and selects 


5 


LI 


L6 


L5 and decision same making and (evaluat$ or estimat$ or determin$) and 


11 


L6 


sort$ and interval $ and rank$ and scorS 


L5 


706/45-47 eels 




T < 

l-ij 


L4 


L,3 and 5?ort$ and intervals and rankU and ^rnrt 




T 4 


L3 


L2 and (evaluatS or estimat$ or determinS) 


155 


L3 


L2 


LI and decision same making 


157 


L2 


LI 


706/46.ccls. 


521 


u 



END OF SEARCH HISTORY 



http://westbrs:9000/bin/cgi-bin/PreSearch.pl 
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Search Results - 



Terms 


Documents 


L19 and @pd<"20031029" 


13 



Database: 



Search: 



US Pre-Grant Publication Full-Text Database 

US Patents Full-Text Database 

US OCR Full-Text Database 

EPO Abstracts Database 

JPO Abstracts Database 

Derwent World Patents Index 

IBM Technical Disclosure Bulletins 



L22 



^^^^^ 



Search History 



DATE: Wednesday, March 29, 2006 Printable Copy Create Case 
Set 

Name Query 

side by 
side 

DB^PGPB,USPZUSOQEPAB,JPAB,DWPlTDBD; PLUR=NO; OP=OR 



Hit 
Count 



Set 
Name 
result 

set 



L22 


L19 and @pd<"2003 1029" 


13 


L22 


L21 


L 1 9 and patient same (choo$ or choic$) 


1 


L21 


L20 


LI 9 and patient near (choo$ or choic$) 


0 


L20 


L19 


therapeutic near system and choic$ and scor$ and rank$ 


35 


L19 


L18 


L 1 7 and choic$ and scor$ 


19 


L18 


L17 


therapeutic near treatment near regimens 


103 


L17 


L16 


L14 and @pd<"2003 1029" 


417 


L16 


L15 


L14@pd<"20031029" 


32788637 


L15 


L14 


LIS and decision and preferences 


706 


L14 


LIB 


questions and answerS and choiceS and evaluatS and scor$ and (sort$ or 
class$) 


2533 


L13 


L12 


Lll and sorting near preferences 


2 


L12 


Lll 


decision same making and choices 


4441 


Lll 
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T 1 A 

LIO 


L9 and @pa< 2000 1 23 1 


16 


LIO 


IS. 


L8 and processor and memory and computer 


190 


L2 


L8 


decision same making and (evaluat$ or estimat$ or determinS) and sort$ 
and mtervalS and rank$ and scorS and (preferencS or choice or choos$) 


311 


L8 


L7 


L4 and selectS 


5 


L7 


L6 


L5 and decision same making and (evaluatS or estimatS or determm$) and 

oV/Ilt4' CUXU 111 l&l veil vP CUlU lClllA.4> CUlU dL^V/lij) 


11 


L6 


L5 


706/45-47 ccis 


1 Ji J 


T C 


T A 


Li J ana son$ ana iniervaikb ana ranK9> ana scorcp 


5 


T A 

L4 


L3 


L2 and (evalnatS or estimat$ or determinS) 


155 


L3 


L2 


LI and decision same making 


157 


L2 


Li 


706/46.ccls. 


521 


LI 



END OF SEARCH HISTORY 



ht^://westbrs:9000/bin/cgi-bin/PreSearch.pl 
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Search' Results 



Terms 


Documents 


L24 and evaluat$ 


85 



Database: 



US Pre-Grant Publication Full-Text Database 

US Patents Full-Text Database 

US OCR Full-Text Database 

EPO Abstracts Database 

JPO Abstracts Database 

Derwent World Patents Index 

IBM Technical Disclosure Bulletins 



Search: 



L25 



r, Interrupt 



Search History 



DATE: Wednesday, March 29, 2006 Printable Copy Create Case 



Set 

Name Query 
side by 
side 

DB=PGPB,USPT.USOC,EPAB,JPAB,DWPI,TDBD; PLUR^NO; OP^OR 



Hit 
Count 



Set 
Name 
result 

. set 



L25 


L24 and evaluat$ 


85 


L25 


L24 


L23 and patient$ and selecting 


89 


L24 


L23 


on near line medical near system$ and cho$ and scor$ and rank$ 


1582 


L23 


L22 


L19 and @pd<"2003 1029" 


13 


L22 


L21 


L19 and patient same (choo$ or choic$) 


1 


L21 


L20 


LI 9 and patient near (choo$ or choic$) 


0 


L20 


L19 


therapeutic near system and choic$ and scor$ and rank$ 


35 


L19 


L18 


L 1 7 and choic$ and scor$ 


19 


L18 


L17 


therapeutic near treatment near regimens 


103 


L17 


L16 


L14 and @pd<"2003 1029" 


417 


L16 


L15 


L14@pd<"20031029" 


32788637 


L15 


L14 


L 1 3 and decision and preferences 

questions and answerS and choiceS and evaluatS and scor$ and (sort$ or 


706 


L14 
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L13 








L12 


\A\ Rnd- sortinff near nreference^ 




T 19 


Lll 


Qccision Same maKing onu cnoices 


4441 


Lll 


T 1 A 
LIU 


T n J . J^ll'^ AAA1 '^'^ 1 It 

Ly and @pa< 20001231 


16 


LIO 


L9 


L8 and processor and memory and computer 


190 


L9 


L8 


decision same makmg and (evaluatS or estimatS or determinS) and sort$ 


311 


L8 


and intervals and rankS and scorS and (preferencS or choice or choosS) 


T ^7 

L7 


L4 and selects ^ 


5 


L7 


L6 


L5 and decision same making and (evaluatS or estimatS or determinS) and 


11 


L6 














L3 ana sorto) ana mtervaiJh ana ranKj ana scorcb 


5 


L4 


L3 


L2 and (evaluatS or estimat$ or determinS) 


155 


L3 


L2 


LI and decision same making 


157 


L2 


Li 


706/46.ccls. 


521 


LI 



END OF SEARCH fflSTORY 



http://westbrs:9000/bin/cgi-bin/PreSearch.pl 
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Search Results - 



Terms 


Documents 


L25 and @pd<"20031029" 


48 



US Pre-Grant Publication Full-Text Database 

US Patents Full-Text Database 

US OCR Full-Text Database 

EPO Abstracts Database 

JPO Abstracts Database 

Derwent World Patents Index 

IBM Technical Disclosure Bulletins 



Database: 



Search: 



L26 




Interrupt 



Search History 



DATE: Wednesday, March 29, 2006 Printable Copy Create Case 



Set 

Name Query 

side by 
side 

DB=PGPB,USPT,USOC.EPAB,JPAB.DWPI,TDBD; PLUR=NO; OP=OR 



Hit 
Count 



Set 
Name 
result 

set 



L26 


L25 and @pd<"2003 1029" 


48 


L26 


L25 


L24 and evaluat$ 


85 


L25 


L24 


L23 and patient$ and selecting 


89 


L24 


L23 


on near line medical near system$.and cho$ and scor$ and rank$ 


1582 


L23 


L22 


L 1 9 and @pd<"2003 1 029" 


13 


L22 


L21 


LI 9 and patient same (choo$ or choic$) 


1 


L21 


L20 


L19 and patient near (choo$ or choic$) 


0 


L20 


L19 


therapeutic near system and choic$ and scor$ and rank$ 


35 


L19 


L18 


L 1 7 and choic$ and scor$ 


19 


L18 


L17 


therapeutic near treatment near regimens 


103 


L17 


L16 


L14 and @pd<"2003 1029" 


417 


L16 


L15 


L14@pd<"20031029" 


32788637 


L15 


L14 


L 1 3 and decision and preferences 


706 


L14 
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L13 


questions and answerS and choiceS and evaliiatS and scor$ and (sort$ or 


2533 


LIS 


L12 


T 11 JinH Qortincy npjir "nrpflF^rf^np^c 


Z 


T 1 O 

Liz 


Lll 


decision same making and choices 


4441 


Lll 


T 1 r\ 

LIO 


L9 and @pd< 2000 1231 


16 


LIO 


L9 


L8 and processor and memory and computer 


190 


L9 


L8 


decision same making and (evaluat$ or estimat$ or determinS) and sortS 


311 


L8 


and mtervalS and rankS and scor$ and (preferencS or choice or choosS) 


L2 


L4 and selects 


5 


L7 


L6 


L5 and decision same making and (evaluatS or estimatS or determmS) and 


11 


L6 


owi ig) oxwx iiiL^i va-ikP oxysx iaiii\.t4> uiiu. odJlkj) 


T 


706/45-47 crk 


IDl J 


T C 


T A 


jlj ana sonj^ anc mtervai^^ ana ranK^ ana scorj 


5 


T A 

L4 


L3 


L2 and (evaluat$ or estimat$ or determin$) 


155 


L3 


L2 


LI and decision same making 


157 


L2 


LI 


706/46.ccls. 


521 


LI 



END OF SEARCH HISTORY 



http://westbrs:9000/bin/cgi-bin/PreSearch.pl 
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Search Results 



Terms 


Documents 


L37 and interval 


45 



Database: 



US Pre-Grant Publication Full-Text Database 

US Patents Full-Text Database 

US OCR Full-Text Database 

EPO Abstracts Database 

JPO Abstracts Database 

Derwent World Patents Index 

IBM Technical Disclosure Bulletins 



Search: 



L38 



m 



iioteiiiiu pti 



Search History 



DATE: Wednesday, March 29, 2006 Printable Copy Create Case 



Set 

Name Query 

side by 
side 

DB=PGPB.USPT.USOC.EPAB,JPAB.DWPI.TDBD: PLUR=NO; OP=OR 



Hit 
Count 



Set 
Name 
result 

set 



L38 


L37 and interval 


45 


L38 


L37 


L36 and @pd<"20031029" 


115 


L37 


L36 


medical and treatment and histograms and (cho$ or select$) and rank$ and 
scores 


• 278 


L36 


L35 


histograms and choices and rankS and scoreS and evaluator 


4 


L35 


L34 


interval near sizes with choice 


2 


L34 


L33 


interval with choice 


1728 


L33 


L32 


L29 and (interval or space) 


16 


L32 


L31 


L29 and interval or space 


3082510 


L31 


L30 


L28 and sort 


12 


L30 


L29 


L28 and rank 


21 


L29 


L28 


L26 and (sortS or rankS) 


48 


L28 


L27 


L26 and (sortS or rankS) 


48 


L27 
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L26 


L25 and (2)od<"20031029" 


4R 




L25 


L24 and evaluatS 


O J 




L24 


L23 and natientS and selecting 


oy 




L23 


on near line medical near svstemS and cho? and ^corS* and rank? 






L22 


LI 9 and ra)od<"20031029" 


1 % 


T 99 


L21 


L19 and natient same fchooS or chnin?'^ 


1 

i 


T 91 


L20 


LI 9 and patient near (choo$ or choic$) 


0 


T 90 


L19 


theraneutic near svstem and choic$ and scor^ and rank"? 

U.X WX Ml./ >^ M VX%/ XXWMX ^ T ^ b WXXX CU.X\X WXX^/XW^/ CU.XVX Ow^/Xk4' CUXVi 1 il 1 1 1\ t|? 




T IQ 


L18 


LI 7 and choic$ and scorS 

X t \/XX\/X WkJ/ CUXVX vX ^ 


1Q 
1 y 


T IR 
1^1 o 


L17 


theraneutic near treatment near regimen ^ 

UXWXOi^WLXLXW XXWCU UVMUXAwXAIf lAVCU. IWgUAlWliD 




T 17 


L16 


L14 and @pd<"2003 1029" 


417 


L16 


T 1 ^ 
LID 


Lr i 4 (gjpa< zuu J 1 uzy 


32788637 


L15 




LIS and decision and preferences 


706 


L14 


L13 


questions and answerS and choice$ and evaluatS and scor$ and (sort$ or 
class$) 


2533 


L13 


L12 


Lll and sorting near preferences 


2 


T 19 


Lll 


decision same making and choices 


4441 


Lll 


T in 


juy ana i^pcx^ zuuuizji 


16 


LIO 


T o 


L8 and processor and memory and computer ^ 


190 


L9 


L8 


decision same making and (evaluatS or estimatS or determinS) and sort$ 
and intervals and rank$ and scor$ and (preferencS or choice or choosS) 


311 


L8 


T n 


L4 and selects 


5 


L7 


L6 


L5 and decision same making and (evaluatS or estimatS or determinS) and 
sorts and intervals and rankS and ^cnrS 


11 


L6 


L5 


706/45-47 eels 


I J ID 




T 4 


ljd dliu auriij) axiu inicrvdio^ duu rdiiK4> onu scorg> 


c 
J 


T A 

L4 


L3 


L2 and (evaluatS or estimatS or determinS) 


155 


L3 


L2 


LI and decision same making 


157 


L2 


LI 


706/46.ccls. 


521 


LI 



END OF SEARCH HISTORY 
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Search Results - 



Terms 


Documents 


L41 and @pd<"20031029" 


46 



Database: 



us Pre-Grant Publication Full-Text Database 

US Patents Full-Text Database 

US OCR Full-Text Database 

EPO Abstracts Database 

JPO Abstracts Database 

Derwent World Patents Index 

IBM Technical Disclosure Bulletins 



Search: 



L42 




pv,;-'MnterruptJ^ 



Search History 



DATE: Wednesday, March 29, 2006 Printable Copy Create Case 

Sfit Hi* Set 

Name Qnfirx Manm 

side by iJUml j.^^,^ 

side set 

DB=PGPB, USPT, USOC,EPAB.JPAB.DWPI,TDBD: PLUR=NO; OP=OR 



L42 


L41 and @pd<"2003 1029" 


46 


L42 


L41 


L40 and scor$ and rank$ and intervals 


75 


L41 


L40 


evaluation near system and choicS 


1033 


L40 


L39 


scoring near choices 


8 


L39 


L38 


. L37 and interval 


45 


L38 


L37 


L36 and @pd<"20031029" 


115 


L37 


L36 


medical and treatment and histograms and (cho$ or selectS) and rank$ and 

scores 


278 


L36 


L35 


histograms and choices and rankS and scoreS and evaluator 


4 


L35 
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Abstract 

A measure of the amount of disagreement, D, in a 
group facing a problem with limited solution 
choices is proposed. D is simple to calculate, 
meaningfully derived and provides a standard 
scale from 0 to 1 for the disagreement of any size 
group facing a number of solution choices. It also 
provides a related measure, d, which allows the 
measurement of the disagreement of each 
individual in the group, D essentially compares the 
number of differences found between pairs of 
individuals in the group with the number of 
differences theoretically possible. Extension of the 
measure to the case where the solution choices are 
represented by ranked, interval and ratio scale 
data shows that D is equal to twice the variance of 
the solution scores, although in this case the 
maximum value ofD may be greater than L The 
properties of this measure are explored and found 
to be similar to what is expected of a measure of 
disagreement. An example application is given, 
illustrating how disagreement at both the 
individual and group levels can be meaningfully 
and usefully represented by d and D. 

Introduction 

In the study of groups* generating group agreement 
and decision acceptance can be as important as 
generating a decision solution [1, 2]. The 
measurement of agreement is therefore an 
increasingly important subject. A simple way to 
measure agreement is in terms of commonality, or 
the number of people who have the same idea [3, 
p364]. This measure however ignores the amount 
of disagreement among the remainder of the group, 
giving only the agreement for one solution option. 



Another method is to instruct the group to reach 
consensus or unanimity and to count the percentage 
of unanimous groups [4]. Such methods ignore the 
varying degrees of agreement possible in groups 
who achieve less than complete unanimity. Recent 
experiments with electronic groups have used a 
more sensitive measure of group agreement [5-7] 
derived from the mathematics of fiazzy set theory 
[8], and calculated by means of a computer 
program. HoWever this method only works with 
interval data, not nominal or ordinal data, such as 
multi-choice questions [9], This paper outlines an 
alternative, simpler measure, which can be applied 
to interval, ordinal and nominal data, and was used 
by the first author in an experiment with electronic 
groups facing multi-choice questions [10]. 

Definition 

It was found to be more natural to develop a 
measure of disagreement rather than agreement, as 
has previously been done, although the two 
obviously relate. Disagreement was conceptualized 
in terms of the square of the distance apart of the 
positions (Rj, Rj) held by two group members. If 
two members hold the same position, their distance 
apart is zero, and their disagreement is zero. The 
situation under consideration is where N (N > 1) 
group members face a problem with K (K > 0) 
mutually exclusive response options (A, C, ... ). 
The disagreement between any two group miembers 
is •ij = (Ri -Rj)2 

The disagreement (d{) between one person who 
chooses option i and the rest of the group can be 
defined as the sum of the disagreement between 
that person and each of the other group members, 
divided by the possible number of relationships: 



Many thanks to Ray Littler of Waikato University for pointing out the connection between our measure of disagreement and 
Simpson's measure of ecological diversity. 
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where fj is the number of people who chose option 
j. If all other participants choose the same option, 
then d s 0 (no disagreen>ent), whereas if everyone 
chooses a different option then d = 1, maximum 
disagreement. 

For nominal data, if two people choose different 
options their disagreement, then •ij ± 1. In this 
case, if Nj group members choose option i, then 
the disagreement of one individual choosing option 
i is the number of disagreements they have with the 
rest of die group (N - Nj) divided by the number 



possible disagreements (N - 1), so dj = 



N-1 



Table 1 gives an example of how individual 
disagreement could be measured for a group of five 
members (N=5) given four solution choices (K=4), 
namely A, B, C and D. The disagreement of the 
individual is the number of others who disagree 
witii them, divided by the maximum number of 
disagreements. 









0.0 


A 


AAAA 


0.25 (1/4) 


A 


AAAB 


0.5 (2/4) 


A 


AACD 


0.75 (3/4) 


A 


ABBC 


1.0 (4/4) 


A 


BCCD 


1.0 (4/4) 


A 


BBBB 



Table 1. Individual disagreement values for 
NssS subjects and K=4 options 

The disagreement for the group (D) can be 
obtained by averaging the disagreement of its 
members: 



For nominal data this becomes 

Xn, n'-n 



D = 



Where N = ^N, and the minimum value of D is 



0, when all members of the group agree. 
Table 2 shows an example of possible group 
disagreement (D) values. As can be seen the 
maximum D value of 1.0 (everyone disagrees) is 
not possible here, because there are five group 
members but only four choices. The line indicates 
where the group moves fi-om majority agreement to 
being unable to make a majority decision. 
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0.0 

0.4 (8/20) 
0.6 (12/20) 
0.7 (14/20) 


Unanimous AAAAA 
All but one AAAAB 
3-2 split AAABB 
3-2 majority AAABC 


0.8 (16/20) 
0.9 (18/20) 


Hung group AABBC 

Maximum AABCD 
disagreement 



Table 2. Group disagreement measure for N=5 
andK=4 

This definition of D can be compared to the use of 
the index of the actual number of mutual 
friendships in a group divided by the number of 
possible mutual friendships as "one of the best 
indicators of a group's cohesion" [11, pl23], 
although in this paper what is being considered is 
solution choice, not friendships formed. D 
measures how much disagreement occurs relative 
to how much is theoretically possible. 

The maximum value of D is the maximum 
separation of subjects, which is not necessarily the 
same as the maximum polarization. For example a 
group response of AAABB would show maximum 
polarization (into two groups), while a response of 
AABCD shows maximum disagreement (of 
everyone). 

The relationship between D and variance 

For the numeric response case the value of D can 
be shown to be directly related to the variance of 
the responses: 



X) ^^^^ ^j^^ 



= N.(N_1) 2N(N.l)s* « 1? 

In other words the group disagreement is equal to 
twice the variance. This provides some basis for 
confidence in D, since variance seems a reasonable 
and reliable measure of disagreement for numeric 
data. 

Maximum value of D 

The maximum disagreement of 1.0 is only possible 
if the number of people in the group is less than or 
equal to the number of choices (N ^ K), when it is 
possible for everyone to disagree. If the number of 
choices is less than the number of group members 
(N > K) it is not possible for everyone to disagree. 
In this case the maximum group disagreement D is 
less than 1. 



The maximum value for D is attained when the 
group is spread as evenly as possible over all K 
options. Suppose r is the integral quotient and a is 
the remainder when N is divided by K, so that N = 
rK + a. Then D will be maximized when all the Nf 
values are as close to r as possible. 

Thus let Ni = r for i = 1, 2, ... K - a and Nj = r + 1 

for i = K - a + .... K. Then the maximum value of 
D will be: 



N"--(K-a)r"-a(r + l)^ 



If N ^ K then obviously D^^j^ = 1, however if N > 
K then the maximum value of D is less than 1. This 
reflects the fact that if there are less choice options 
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than people in the group some people must agree 
(select the same choice option). Table 3 shows how 
the maximum value of D reduces from 1.0, as the 
group size increases, for selected vales of K. In 
general, as N gets very large, tends towards 1 
- 1/K. For example in the case where there are two 
solution choices (K = 2), as N becomes very large, 
l^max t^iids to 0.5. This suggests that large groups 
facing two choice problems can experience only 
half the disagreement that can occur for small 
groups facing the same choices. 




2 


1.000 


1.000 


1.000 


1.000 


3 


0.667 


1.000 


1.000 


1.000 


5 


0.600 


0.800 


1.000 


1.000 


10 


0.556 


0.733 


0.889 


1.000 


100 


0.505 


0.673 


0.808 


0.909 


1,000 


0.501 


0.667 


0.801 


0.901 


1,000,000 


0.500 


0.667 


0.800 


0.900 



Table 3. Maximum D by increasing N for 
various K 



Discussion 

The advantages D and d as measures of group and 
individual disagreement are: 

1. Simple. D and d are simple enough to be 
calculated by hand for small groups. 

2. Sensitive* As can be seen from Table 2, D 
recognizes that a group response of AAABC 
(D = 0.7) shows more disagreement than a 
group response of AAABB (D = 0.6). 

3. Valid. D and d are derived from a definition of 
the disagreement between individuals which is 
meaningful in terms of what is normally 
understood to be disagreement. 



4. Scaled. For nominal data D and d have a fixed 
scale, from 0 (unanimity) to 1 (everyone 
disagrees) regardless of group size, although 
the maximum value of D is less if there are 
more people than choices. 

5. Related. There are advantages in group 
research having related measures of individual 
and group disagreement. 

It is an interesting question whether the properties 
of D relate to what is known about group 
disagreement, and whether the measure itself has 
any implications in this area. For example the 
reduction of Djoux ^ ffoup size increases, shown 
in Table 3, suggests that for a problem with a 
relatively low number of solution choices, 
increasing group size decreases the maximum 
disagreement possible in the group. Since D is the 
average of d, a possible benefit of larger decision 
groups is a reduction in the maximum individual 
and group disunity that can occur. While from a 
task result perspective, the optimal group size for 
face-to-face interactions may be as low as five or 
six [12], it may be that larger groups are better 
from a social, group agreement perspective, 
because they have less potential for disagreement, 
especially for problems with few choices. Larger 
groups can handle limited choice problems with 
less threat to group unity. 

Table 1 shows that when a group of five moves 
from consensus to one person disagreeing, the 
dissenting individual's disagreement changes fix^m 
d = 0.0 to a maximum of d = 1.0. This seems to 
reflect the nature of the situation. To break 
unanimity an individual must disagree with all the 
other members of the group, and it is not possible 
for them to disagree with only one or two of the 
others. To effect such a major change can be 
expected to be difficult to do, as it involves a 
change from no disagreement to maximum 
disagreement. Table 2 shows that for the consensus 
group as a whole, when one person disagrees, the 
group D registers a change of almost half the scale 
(D = 0.4), suggesting that for a group of this size 
the disagreement of one person is a major event. 
This effect however reduces as group size 
increases. For a group of ten, one person 
disagreeing gives a D value of 0.2, and for a group 
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of 100 the D value is only 0.02. Thus the larger the 
group, the less important one person disagreeing is, 
which seems reasonable. There seems to be an 
certain inherent stability in consensus, especially 
for small groups. 

D is a function not only of the group size but also 
the number of solution choices. The number of 
solution choices can be taken to be a measure of 
the complexity of the problem. More complex 
problems provide more choices. For a given size 
group, the more choices confronting it the more 
disagreement can occur, up to the point where there 
are as many choices as there are members in the 
group. Equivocal problems [13] are problems 
which are ambiguous - it is not even clear what the 
problem is. Could such problems be taken to be the 
special case where the number of solution options 
is infinitely large? Such problems could generate 
maximum disagreement in any size group. 

Ecological diversity 

An equivalent formula to that derived here can be 
found in mathematical ecology, where a given 
habitat can have N creatures and K species. If all N 
creatures are of the same species (K = 1) then the 
ecological diversity is low, whereas if every animal 
is a different species (K = N) then the ecological 
diversity is high. Simpson's measure of ecological 
diversity [14, p223] applies to the case of N 
animals of which Nj belong the j'th species (j = 
1,2, ... K). The probability of choosing one animal 
without replacement from species j is (Nj / N), and 
the probability of choosing a second from the same 
species is (Nj - 1)/ (N - 1). Sunmiing the 
probability of choosing two animals of the same 
species over all species and subtracting from 1 
gives Simpson's measure: 

iSJc N(N-l) 
which may be rearranged to give 



which is the same as the group disagreement 
measure defined earlier for the nominal case. It is 
interesting that two situations so different as group 
disagreement and ecological habitat diversity can 
give rise to the same formula, derived in different 
ways. The measure is the same, although the 
situations seem different. For example subjects 
asked to choose again may show a test-retest 
correlation less than one, while animals by contrast 
do not change their species on re-sampling. There 
may be a higher concept that incorporates both 
group disagreement and ecological diversity in the 
same theoretical framework, where the species of 
an animal equates to the choice of an individual. 

ProbabiUty distribution 

Assuming all solution options are equally likely to 
be chosen by all group members gives the 
probability distribution as shown in Table 4 for N = 
5 and K = 4. The distribution of D is positively 
skewed, suggesting that there are more ways a 
group can disagree than there are ways they can 
agree. The mean value of D given the distribution 
shown is 0.75, which is half way between the 
smallest majority (a 3-2 majority) and a hung 
group, as represented by the line in Table 4. This 
line represents the point at which the group loses 
the ability to form a majority decision. 



agreement 1 


response 






0.0 


Unanimous 


AAAAA 


4/1024 


0.4 (8/20) 


All but one 


AAAAB 


60/1024 


0.6(12/20) 


3-2 split 


AAABB 


120/1024 


0.7 (14/20) 


3-2.majority 


AAABC 


240/1024 


0.8(16/20) 


Hung group 


AABBC 


360/1024 


0.9 (18/20) 


Maximum 
disagreement 


AABCD 


240/1024 



Table 4. Probability distribution for D (N=5 
andKa4) 



A distribution such as shown in Table 4 can be 
used to defme a null hypothesis that all solution 
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options are equally likely. It represents a situation 
where subjects have no reason to select one option 
over another, either because the problem is very 
difficult, and subjects do not know which option is 
correct, or because subjects are given choices they 
find equally attractive. By contrast, for an easy 
problem such as 2 + 2 = ? the correct answer will 
have a high probability and the others very low 
ones. The same situation would arise when subjects 
had a strong preference or bias for one option. 

This null hypothesis does not recognize the 
distinction between intellective problems (which 
have a right/wrong answer) and preference 
problems (whose answer depends on user 
preference), as defined in McGrath*s task 
circumplex [15]. An even distribution of the 
probability of selecting solution options could occur 
either because subjects found an intellective problem 
very difficult, or because they had no particular bias 
on a preference problem. The key factor is whether 
all response options have equal probability. Indeed 
from a subject's point of view, there may be very 
little difference between a difficult intellective 
problem and a preference problem, where subjects 
choose the option they prefer. The response 
probabilities of the subjects can be used to define 
the choice situation, rather the experimenter defined 
task context. 

Example application 

This measure was used in an experiment where 
computer-mediated groups of five subjects, 
interacting only through a computer network, had 
to provide group solutions to multi-choice 
questions witii four choice options [10], The 
treatment group first chose their answer without 
seeing how others voted. On their second vote 
round however they could see how the group voted 
on the first vote, and likewise on the third vote 
round they could see the second round votes. The 
control group simply voted alone without knowing 
how the others voted. The treatment situation was 
designed to minimize personal and informational 
influence, while retaining the necessary conditions 
for normative influence. Subjects could not discuss 
the questions at all, and all responses were 
anonymous. The main dependant measure of the 
experiment was group disagreement (D), which 



showed a highly significant effect (F = 242.6, p < 
0.000***). The disagreement measures made it 
possible to analyze the data in further detail. Before 
their second vote subjects could find themselves in 
the minority (d ^ 0.75) or in the majority (d < 
0.75). It was expected that those in the minority 
would tend to change their vote and those in the 
n^jority would not. 

The results, shown in Table 5, indicate a clear trend 
in the expected direction, with a threshold effect 
once the voter moves into a minority, as has been 
found in other studies [16]. Individuals in a 
minority seem to be influenced by the rest of the 
group. However individual disagreement (d) alone 
does indicate whether the rest of the group agreed 
or disagreed among themselves. For example if an 
individual disagrees with everyone else, then they 
are in a minority, but if the rest of the group 
disagrees with each other as well, then everyone is 
in the minority. In this case there is no majority to 
attract the individual to change position. It was 
proposed that an individual disagreeing with the 
rest of the group would be more likely to change 
their position if the rest of the group agreed among 
themselves, and a group disagreement score was 
c£ilculated/(?r the other four members of the group, 
called DRest. Table 6 shows the possible 
combinations of d and Drest and in each cell gives 
an example of the vote situation, e.g. AAAAB 
describes the case where the individual (shown in 
bold) finds that one person in the group disagrees 
with them. As can be seen from Table 6, not all 
combinations of d and Drest are possible. 

Table 7 shows the percentage who changed their 
vote position in each situation, and the number of 
times that combination occurred (in brackets). 
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iMl^iiiiSi 


% changed vote 
N 


0.8% 2.0% 7.1% 

479 403 368 


363% . 716% 



Table 5. Percentage of changed votes by prior individual disagreement 









0.0 Disagree with no-one 
0.25 Disagree with one 
0.5 Disagree with two 


AAAAA 

AAAAB 

AAABB AAABC 


0.75 Disagree with three 
1.0 Disagree with aU 


BAAAB BAABf BABC3| 
DMAA DAAAB PAABB DAA^G 



Table 6. Examples of individual by rest of group disagreement combinations 



: ' h. , ' ^ •„ '■ ' ; 


Rest of group disagreement (Brest) ;^ 


0.6 M/^:.. 0.67 , , 0.83 "'ItSsi J 


0.0 Disagree with no-one 
0.25 Disagree wifli one 
0.5 Disagree with two 


1.0% 

(1625) 

2.9% 
(886) 

83% 3.8% 

, „..C2«9) , (261) 


0.75 Disagree with three 
1.0 Disagree with all 


-•■■■•45.0% ■ 253%' '"USf^ ] 
■■: (238) (190) . 0^ 
■ 66.1% 68.4% 77.0% 603% 

:„(369) . (247i...^,„;,(6l);, .M.. ...c. 1 



Table 7. Vote change by vote situation 
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The results in Table 7 follow an interesting pattern. 
Firstly 83.6% of all vote events involved no change 
in vote position, so subjects tended to stay with 
their previous vote position. For d = 0.5 and d = 
0.75 the percentage who changed position 
decreased as the others in the group disagreed 
more, as expected. However when d = 1.0 and the 
subject disagrees with everyone, a majority of four 
in agreement against the subject produces no 
greater effect than a majority of three, perhaps 
indicating some sort of threshold has been reached. 
The case DAABB is interesting in that it involves 
two other competing candidates for the group 
majority, and produced the highest vote change. If 
the individual is identified with the group, and if 
the group needs agreement, then the likelihood of 
vote change can be taken as the probability that the 
individual* s solution choice will form a majority, 
compared to the probability that another solution 
choice can form a group majority. These ideas can 
be put as propositions: 

L Individuals mil tend to maintain their 
previously adopted position, 

2. The likelihood of no change depends on the 
probability that the individual* s current 
position will form a group majority, with a 
threshold effect occurring at the 
minority/majority boundary, 

3. The likelihood of change to a particular 
alternative depends on the probability that the 
alternative will form a group majority, and, 

4. A small amount of change will occur 
regardless. 

These propositions could form the basis of a 
computer simulation of normative group behavior, 
using parameter values suggested by the results 
given. For example the probability of remaining 
with the previous position regardless seems to be 
about 33%, the random change factor about 1%, 
the majority-minority threshold from 0% to about 
4S%, and the likelihood of change induced by 
altemate positions from 2% to about 20%, 
depending on group situation. Such a program 
could operate dynamically, and simulate not only 
computer-mediated normative effects [17, 18] but 
also the sequence effects that occur when 
individuals respond in any order [19], The 



measures proposed can be used to summarize large 
amounts of data in a meaningful way, and may be 
useful in theoretical models. 

Conclusions 

Measures of group and individual disagreement, D 
and have been proposed which are easy to 
calculate, meaningfully derived and appear to have 
the properties desired. For numeric data D is a 
simple transformation of the response variance. D 
and d provide sensitive measures of disagreement 
in groups facing problems with a given number of 
solution alternatives. They are also meaningfully 
related, as the group disagreement D is the average 
of the group member disagreements, d. As this 
paper has presented only a cursory analysis of D 
and d there may be problems in their application. 
However at this stage they appear to be a useful 
ways of measuring an increasingly important 
product of group activity - the amount of 
disagreement in the group. The measurement of 
disagreement in groups of varying size, facing 
problems with varying numbers of solution 
choices, could be a rich area for future research. 

References 

1 . Maier, N.R.F., Problem Solving 
Discussions and Conferences, 1963: McGraw-Hill, 
New York. 

2. Boje, D.M. and J.K. Mumighan, Group 
confidence pressures in iterative decisions. 
Management Science, 1982. 28(10): p. 1187-1197. 

3. Lorge, I., et al^ A survey of studies 
contrasting the quality of group performance and 
individual performance. Psychological Bulletin, 
1958, Nov. 55(6): p. 337-372. 

4. Sniezek, J., Groups under uncertainty: an 
examination of confidence in group decision- 
making, organizational behaviour and human 
decision processes, 1992. 52: p. 124-155. 

5. Sambamurthy, V, and W. Chin, 77i^ effects 
of group attitudes towards alternative GDSS 
designs on the decision making performance of 
computer supported groups. Decision Sciences, 
1994.25(2): p. 215-241. 

6. Tan,B.C.,K.Wei,andS.R. 
Krishnamurthy . Effects of support and task type on 
group decision outcome: A study using SAMM. in 



358 



Proceedings of the 24th Hawaii International 
Conference on System Sciences. 1 99 1 . 

7. Watson, RT., G, DeSanctis, and M.S. 
Poole, Using a CDSS to facilitate group consensus: 
Some intended and unintended consequences. 
Management Information Systems Quarterly, 1988, 
Sep. 12(3): p. 463-478. 

8. Spillman, B., R. Spillman, and J. Bezdek, A 
fuzzy analysis of consensus in small groups, in 
Fuzzy Sets: Theory and Application to Policy 
Analysis and Information Systems^ P.P. Wang and 
S.K. Chang, Editors. 1980, Plenum: New York. p. 
291-308. 

9. Tan, B.C.Y., H.H. Teo, and K.K. Wei, 
Promoting consensus in small decision making 
groups. Information and Management, 1995. 28(4): 
p. 251-259. 

10. Whitworth, B., Generating Group 
Agreement and Decision Confidence in Computer 
Mediated Groups, PhD Thesis in Department of 
Management Systems, 1997, University of 
Waikato, Hamilton, New Zealand. 

1 1 . Dimock, H.G., Groups: Leadership and 
Group Development, 1986, San Diego: University 
Associates. 

1 2. Shaw, M., Group Dynamics: The 
Psychology of Small Group Behaviour. Vol. 3rd 
Edition. 1981: McGraw-Hill, New York. 

13. Daft, R.L., R.H. Lengel, and L.K. Trevino, 
Message equivocality, media selection, and 
manager performance: Implications for 
information systems. Management Information 
Systems Quarterly, 1987(Sep): p. 354-366. 

14. Pielou, E.G., An Introduction to 
Mathematical Ecology. 1969: Wiley. Chapter 18. 

15. McGrath, J.E,, Groups: Interaction and 
Performance. 1984: Englewood Cliffs, N.J.: 
Prentice-Hall, Inc. 

16. Hoffman, L.R., The group problem-solving 
process^ in Group Processes, L. Berkowitz, Editor. 
1978, Academic Press, p. 101-113. 

17. Lea, M. and R. Spears, Computer-mediated 
communication, de-individuation and group 
decision making. International Journal of Man- 
Machine Studies. 1991. 34: p. 283-301. 

18. Sia, C. C.Y. Tan, and K.K. Wei, Will 
distributed GSS groups make more extreme 
decisions? An empirical study. Proceedings of the 



17th International Conference on Information 
Systems, Cleveland.Ohio, 1996, Dec 16-18: p. 326- 
338. 

19. McGuire, T.W., S. Kiesler, and J. Siegel, 
Group and computer-mediated discussion effects in 
risk decision making. Journal of Personality and 
Social Psychology, 1987. 52(5): p. 917-930. 



359 



IEEE Xplore# Search Result 



Page 1 of 1 



Xplore 



RELEASE 2.1 



Home I Login | Logout | Access Information | Alerts | 
Welcome United States Patent and Trademark Office 



IQISearch Results 



BROWSE 



SEARCH 



Results for "((decision <near> system <and> choice <and> score <and> rank <and> int...'* 
Your search matched 1 of 1332769 documents. 

A maximum of 100 results are displayed, 25 to a page, sorted by Relevance in Descending order. 



IEEE XPLORE GUIDE 

Qe-rrail 



» Search Options 
View Session History 
New Search 

» Key 

IEEE JNL IEEE Journal or ' 
Magazine 

lEE JNL lEE Journal or Magazine 

IEEE CNF IEEE Conference 
Proceeding 



lEE Conference 

Proceeding 



lEE CNF 
lEEESTD IEEE Standard 



Modify Search 

[((decision <near> system <and> choice <and> score <and> rank <and> lntervaO<in> 



Check to search only within this results set 
Display Format: ® Citation O Citation & Abstract 



x j view selected items ] Select am Deselect am 

Ij 1. Measuring disagreement in groups facing limited choice problems 

Whitworth. B.; Felton. R.; 

System Sciences. 1998.. Proceedings of the Thirty-First Hawaii Internationai C 
Volume 1 , 6-9 Jan. 1 998 Page(s):351 - 359 vol.1 
Digital Object Identifier 10.1 109/HICSS.1998.6531 19 

AbstractPlus | Full Text: PDF(712 KB) IEEE CNF 

Rights and Permissions 



Hinspec' 



Help Contact Us Privacy & : 
©Copyright 2006 lEEE- 



http://ieeexplore.ieee.org/search/searchresult.j sp?query 1 =&scope 1 =metadata&op 1 =and&que.. . 3/29/06 



